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Abstract 

In this paper, we study the sample complexity of probabilistic methods for uncertain systems. In particular, we show the 
role of the binomial distribution for some problems involving analysis and design of robust controllers with finite families. We 
also address the particular case in which the design problem can be formulated as an uncertain convex optimization problem. 
The second main contribution of the paper is to study a general class of sequential algorithms which satisfy the required 
specifications using probabilistic validation methods and, at each iteration of the sequential algorithm, a candidate solution 
is probabilistically validated. The results of the paper provide the sample complexity which guarantees that the obtained 
solutions meet some pre-specified probabilistic specifications. 
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1 Introduction 

Research on probabilistic analysis and design for systems 
and control has significantly evolved. In recent years, 
key areas where we have seen convincing developments 
include uncertain and hybrid systems [31]. A key feature 
of this approach is the use of the theory of rare events 
and large deviation inequalities which suitably bound 
the tail of the probability distribution. These inequalities 
are crucial in the area of Statistical Learning Theory [32] , 
which has been utilized for feedback design of uncertain 
systems [33]. 

Design in the presence of uncertainty is of major rel- 
evance in different areas. Unfortunately, the related 
semi-infinite optimization problems are often NP-hard 
problems, and this may seriously limit their solution 
from the computational point of view [7]. There are 
two approaches to resolve this NP-hard issue. The first 
approach is based on the computation of deterministic 
relaxations of the original problem which are normally 
solved in polynomial time. However, this might lead 
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to overly conservative solutions [27]. An alternative 
paradigm is to assume that the plant uncertainty is 
probabilistically described and a randomized algorithm 
may be developed to compute, in polynomial time, a 
solution with some given properties stated in terms of 
the probability of error [31], [33]. 

A survey on probabilistic methods can be found in [13]. 
Two complementary approaches, non-sequential and se- 
quential, have been proposed. A classical approach for 
non-sequential methods is based upon Statistical Learn- 
ing Theory [32] , [33] . Subsequent work along this direc- 
tion includes [21], [34], [35], [2]. In [4], [3] and [23] the case 
in which the design parameter set has finite cardinality 
is analyzed. The advantage of these methods is that the 
problem under attention may be non-convex. For con- 
vex optimization problems, a successful non-sequential 
paradigm, denoted as the scenario approach, has been 
introduced in [9] and [10], see also [14], [15], [12] and [4] 
for related results. 

For non-sequential methods the original robustness 
problem is reformulated as a single optimization prob- 
lem with sampled constraints which are randomly gen- 
erated. A relevant feature of these methods is that they 
do not require any validation step and the sample com- 
plexity is defined a priori. The main result of this line 
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of research is to derive explicit lower bounds to this 
required sample size. However, the obtained explicit 
sample bounds can be overly conservative because they 
rely on a worst-case analysis and grow (at least linearly) 
with the number of decision variables. 

For sequential methods, the resulting iterative algo- 
rithms are based on stochastic gradient [8], [26], ellip- 
soid iterations [20] , [24] , [29] , or analytic center cutting 
plane methods [11], [28], see also [1] for other classes of 
sequential algorithms. Convergence properties in finite- 
time are in fact one of the focal points of these papers. 
Various control problems have been solved using these 
sequential randomized algorithms, including robust LQ 
regulators [26], switched systems [22], and uncertain lin- 
ear matrix inequalities (LMIs) [8]. Sequential methods 
are mostly used for uncertain convex feasibility prob- 
lems because the computational effort at each iteration 
is affordable. However, they have been studied also for 
non-convex problems, see [2], [19]. 

The common feature of all these sequential algorithms is 
the use of the validation strategy presented in [24] . The 
candidate solutions provided at each iteration of these 
algorithms are validated using a validation set which is 
drawn according to the probability measure defined in 
the uncertain set. If the candidate solution satisfies the 
design specifications for every sampled element of this 
validation set, then it is classified as probabilistic solu- 
tion and the algorithm terminates. The main point in 
this validation scheme is that the cardinality of the val- 
idation set increases very mildly at each iteration of the 
algorithm. The strategy guarantees that, if a probabilis- 
tic solution is obtained, then it meets some probabilis- 
tic specifications. A similar approach, introduced in [17], 
has been presented in [13] in the context of sequential 
algorithms. The contribution is a reduction on the car- 
dinality required for the validation sets. See also [5] for 
related results. 

In this paper, we derive the sample complexity for var- 
ious analysis and design problems related to uncertain 
systems. In particular we provide new results which 
guarantee that the tail of the binomial distribution is 
bounded by a pre-specified value. These results are then 
applied to the analysis of worst-case performance and 
constraint violation. With regard to design problems 
we consider the case of finite families and the special 
case when the design problem can be recast as a robust 
convex optimization problem. 

The second main contribution of this paper is to propose 
a relaxed validation scheme which allows the candidate 
solution to violate the design specifications for one (or 
more) of the members of the validation set. The idea of 
allowing some violations of the constraints is not new 
and can be found, for example, in the context of system 
identification [6], chance-constrained optimization [15] 
and Statistical Learning Theory [2] . This scheme makes 



sense in the presence of soft constraints or when a so- 
lution satisfying the specifications for all the admissible 
uncertainty realizations cannot be found. 

The rest of the paper is organized as follows. In the next 
section, we first introduce the problem formulation. In 
Section 3, we provide bounds for the binomial distribu- 
tion which are used in Section 4 to analyze the proba- 
bilistic properties of different schemes involving random- 
ization. In Section 5, we introduce the proposed family of 
probabilistically validated algorithms. The sample com- 
plexity of the validating sets is analyzed in Section 6. A 
comparison with the validation scheme presented in [24] 
is provided in Section 7. Section 8 studies how to use 
the results of the paper in the context of non-sequential 
randomized algorithms. A numerical example in which 
the different schemes are used to address a robust iden- 
tification problem is presented in Section 9. The paper 
ends with a section of conclusions. 

2 Problem Statement 

We assume that a probability measure Pryy over the 
sample space W is given. Given W, a collection of N 
independent identically distributed (i.i.d.) samples w = 



{w^ ', . . . ,w^ '} drawn from W belongs to the Carte- 
sian product yV^ = Wx---xW{N times). Moreover, 



if the collection w of A^ i.i.d. samples {i 



,(1) 



.,w 



{N) 



} 



is generated from W according to the probability mea- 
sure Pryv, then the multisample w is drawn according 
to the probability measure Pryyjv . The scalars 77 e (0, 1) 
and (5 e (0, 1) denote probabilistic parameters called ac- 
curacy and confidence, respectively. Furthermore, In(-) 
is the natural logarithm and e is the Euler number. For 
a; e R, a; > 0, [x\ denotes the largest integer smaller 
than or equal to x; \x~\ denotes the smallest integer 

greater or equal than x. For a S R, a > 1, ^(a) denotes 

00 
the Riemann zeta function (i.e. ^{a) ~ ^ -^). 

k=l 

In a robustness problem, the design parameters and aux- 
iliary variables are parameterized by means of a decision 
variable vector 9, which is denoted as "design parame- 
ter" , and is restricted to a set 9. Furthermore, the uncer- 
tainty w is bounded in the set W and represents one of 
the admissible uncertainty realizations. We also consider 
a binary measurable function g : x W — >■ {0, 1} and a 
real measurable function / : 8 x W — > R which helps to 
formulate the specific design problem under attention. 
More precisely, the binary function giOxW— >{0,1}, 
is defined as 



l{0,w):-- 



a 9 meets design specifications for w 

1 otherwise. 



where design specifications are, for example, iJoo norm 



bounds on the sensitivity function, see specific examples 
in [31], or approximation of functions, see the numerical 
example in Section 9. 

Given 6* G O, the constraint g{9, w) = is satisfied for 
a subset of W. This concept is rigorously formalized by 
means of the notion of "probability of violation" , which 
is now introduced. 

Definition 1 [probability of violation] Consider a prob- 
ability measure Pryy over W and let 9 <E be given. The 
probability of violation of 9 for the function g : Q xW ^ 
{0, 1} is defined as 



In this setting, we draw TV i.i.d. samples {w'^^'' , • • • , w'-^-' } 
from W according to probability Prw and solve the sam- 
pled optimization problem 



min J{9) subject to g{ 



jW)^0,i^l,...,N. (3) 



Since obtaining a global solution to this problem is still 
a difficult task in the general case, we analyze in this 
paper the probabilistic properties of any suboptimal so- 
lution. Furthermore, if at most m violations of the N 
constraints are allowed, the following sampled problem 
can be used to obtain a probabilistic relaxation to the 
original problem (2) 



E{9) :== Prw { w £ W : g{9, w)^l}. 



Using this notion we study the robust optimization prob- 
lem 



mm J{9) subject to E{9)<f] 



(1) 



where J : — > (— oo, oo) is a measurable function which 
normally represents the controller performance. Given 
accuracy rj <E (0, 1) and confidence 6 G (0, 1), the main 
point of the probabilistic approach is to design an al- 
gorithm such that any probabilistic solution 9 obtained 
running the algorithm satisfies E{9) < rj with probabil- 
ity no smaller than 1 — 5. 

Even in analysis problems when G is given, it is often 
very hard to compute the exact value of the probability 
of violation E{9) because this requires to solve a mul- 
tiple integral with a non-convex domain of integration. 
However, we can approximate its value using the concept 
of empirical mean. For given 9 E 0, and multisample 



{w 



(1) 



,w 



(AT) 



} drawn according to the probabil- 



ity measure Pryyw, the empirical mean oi g{9,w) with 
respect to w is defined as 



E{9,w) :-- 



1 ^ 



.Mh 



Clearly, the empirical meem E{9, w) is a random variable. 
Since (/(•, •) is a binary function, E{9, w) is always within 
the closed interval [0, 1]. 

The power of randomized algorithms stems from the fact 
that they can approximately solve non-convex design 
problems (with no violation) of the type 



min J{9) subject to g{ 



= 0, for ah w eW. 



(2) 



min J{9) subject to 



N 

E 

e=i 



g{9,w 



W) 



< TO. 



(4) 



Randomized strategies to solve problems (3) and (4) 
have been recently studied in [2], see also [31]. In order 
to analyze the probabilistic properties of any feasible 
solution to problem (4), we introduce the definition of 
probability of failure. 

Definition 2 [probability of failure] Given N, rj G (0, 1), 
the integer m where < m < N and g : x W — > {0, 1}, 
the probability of failure, denoted byp{N,r],m) is defined 
as 



p{N, f],m) := Prw" {w G W 



N 



There exists 9 E Q 



such thatE{9,w) < — and E{9) > 77}, 



which is slightly different than the probability of one- 
sided constrained failure introduced in [2]. We notice 
that if the probability of failure p{N, rj, to.) is no greater 
than d then every feasible solution 9 G Q to problem (4) 
satisfies E{9) < rj with probability no smaller than 1 — 5. 
Therefore, the objective is to obtain explicit expressions 
yielding a minimum number of samples N such that the 
inequality p{N, rj, m) < 5 holds. 

We notice that the probability of failure can be easily 
bounded by the binomial distribution if = {9} is a 
singleton, which represents an analysis problem. In this 
case, p{N, rj, m) can be easily obtained as follows 

p{N, rj, m) — 



Prwjv{w G Vy'" : E{9,w) < — and E{9) > rj} = 

N 



Prw"{w G W^ : ^ gi9, w'^^) < m and E{9) > r/} < 

N 

Prw"{wG>V^ : ^g(^,w;(^)) <mand^(^) = r/} = 



€=1 



E 

4 = 



N 



v\i-v) 



N-i 



In Subsection 4.2 we derive bounds on p{N, rj, m) when 
Q consists of a finite number of elements. On the other 
hand, if Q consists of an infinite number of elements, a 
deeper analysis involving Statistical Learning Theory is 
needed [31], [33]. In Subsection 4.3 we study the proba- 
bilistic properties of the optimal solution of problem (3) 
under the assumption that g{9, w) = is equivalent to 
f{d,w) < 0, where f : Q x W ^ R is a. convex func- 
tion with respect to in 0. In this case the result is not 
expressed in terms of probability of failure because it 
applies only to the optimal solution of problem (3), and 
not to every feasible solution. 



Proof: 

The proof of the lemma follows from the following se- 
quence of inequalities: 



B(iV,77,™)=a"^ 



a-"^r,\l-rj) 




N- 



N-i 



3 Sample complexity for the binomial distribu- 



tion 



Given a positive integer N and a nonnegative integer 
m, m < N, and rj £ (0, 1), the binomial distribution is 
given by 



B{N,7j,m):^J2 



N 



r/Xl-J?) 



N-i 



i=0 



The problem we address in this section is the explicit 
computation of the sample complexity, i.e. a function 
N{ri, m, 6) such that the inequality B{N, rj^ra) < 5 holds 
for any N > N{ri,m,6), where 6 € (0,1). As it will 
be illustrated in the following section, the inequality 
B{N,rj,m) < 5 plays a fundamental role in probabilis- 
tic analysis and design methods. Although some explicit 
expressions are available, e.g. the multiplicative and ad- 
ditive forms of Chernoff bound [16] , the results obtained 
in this paper are tuned on the specific inequalities stem- 
ming from the problems described in Section 4. 

The following technical lemma provides an upper bound 
for the binomial distribution B{N, rj, m). 

Lemma 1 Suppose thatrj £ (0, 1) and that the nonnega- 
tive integer m and the positive integer N satisfy m < N . 
Then, 



J^ N\ 



N 

<a™(:i + l-r/) , Va> 1. 



D 



We notice that each particular choice of a > 1 provides 
an upper bound for B{N, rj, m). When using Lemma 1 to 
obtain a specific sample complexity, the selected value 
for a plays a significant role. 

Lemma 2 Given (5 £ (0, 1) and the nonnegative integer 
m, suppose that the integer N and the scalars rj £ (0, 1) 
and a > 1 satisfy the inequality 



N > - ( — ^ I ( In - + m In a 
Then, m < N and 



(5) 



B{N,rj,m)^J2 



N 



r]'{l-T]Y'-' <S. 



Proof: 

We first prove that if inequality (5) is satisfied then N > 
TO. Since rj £ (0, 1) and S £ (0, 1), (5) implies 



N > 

Next, we notice that 

d / a 

■ In a 



a-1 



In a 



In a 



da\^a— 1 J \{a — 1)^ J a — I 

Since In a < a — 1 for every a > 1 , it follows that 



i(^l^'^)>((^)(«-^) + ^^«- 



Using this fact, we conclude that . " j^. In a is a strictly 
increasing function for a > 1. This means that 



N > I In a I ?Ti > lim \n a \ m = m. 



a-1 



a^l V a — 1 



We now prove that (5) guarantees that a™(2 + 1 — 77)^ < 
d. The inequality (5) can be rewritten as 



Ni] > In — h mln a. 

a J S 



(6) 



Since x < — In (1 — x) for every x e (0, 1), and ?7(^— -^) G 
(0, 1), from inequality (6), we obtain a sequence of in- 
equalities 



- A^ In ( 1 — ry ( j j > In — + m In a 



ln(5>?7ilna + A^ln 1 — 77 



S> 



^ + 1 



77 



N 



We have therefore proved that inequality (5) implies m < 
N and a'"(- + 1 — 77)^ < S. The claim of the property 
follows directly from Lemma 1. □ 

Obviously, the best sample size bound is obtained tak- 
ing the infimum with respect to a > 1. However, this 
required to solve a one-dimensional optimization prob- 
lem for given 77, d and 777. We observe that a suboptimal 
value can be immediately obtained setting a equal to the 
Euler constant, which yields the sample complexity 



TV > 



e- 1 



In 



777j. We 

= 1 + 



Since ^ < 1.59, we obtain N > ^ {\n 
also notice that, if 777 > then the choic 

-^^-^ + \ 2-^^—^ provides a less conservative bound at the 
price of a more involved expression. Based on extensive 
numerical computations for several values oiri,S and 777 it 
seems that this bound is very close to the "optimal" one. 
The explicit bound presented in the following Corollary 
constitutes one of the main results of this paper. 

Corollary 1 Given (5 G (0, 1) and the nonnegative inte- 
ger m, suppose that the integer N and the scalar rj e (0, 1) 
satisfy the inequality 



N > 



1 



lni + ^27nlnl 



(7) 



Then, 



N 



B{N,rj,m)^J2\ . ] v'i^ - vf^' < S- 



(8) 



i=0 



The proof of this corollary is shown in the appendix. This 
corollary improves upon the explicit expression obtained 
using the multiplicative form of the Chernoff bound [31], 
which is given by 



N > - 777 + In - 
77 \ 6 



4 Sample complexity for probabilistic analysis 
and design 

We now study some problems in the context of random- 
ized algorithms where one encounters inequalities of the 
form 




B{N,7j,m)=J2 



N 



v^i-vY ' <s. 



1=0 



In particular, we show how the results of the previ- 
ous section can be used to obtain explicit sample size 
bounds which guarantee that the probabilistic solutions 
obtained from different randomized approaches meet 
some pre-specified probabilistic properties. 

4-1 Worst- case performance analysis 

We recall a result shown in [30] for the probabilistic 
worst-case performance analysis. 

Theorem 1 Assume that, given the function f : 
G X W — > R, and 6 G O, the multisample w = 
{w^^' , . . . , w^^'} is drawn from W^ according to proba- 
bility Pryyjv . Suppose also that 



If 



7= max f(§,w^^^). 



In i 

iV> ^ 



In^' 
1-)) 



then Prw{w G W : f{9,w) > 7} < 77 with probability 
no smaller than 1 — 6. 

The proof of this statement can be found in [30] and is 
based on the fact that Prw{7« G W : f{0, 777) > 7} < 77 
with probability no smaller than 1 — (1 — rj) . Therefore, 
it suffices to take TV such that S (TV, 7/, 0) = (1-77)^ < 5. 



4-. 2 Finite families for design 

We consider the non-convex sampled problem (4) for the 
case when Q consists of a set of finite cardinality nc- 
As a motivation, we study the case when, after an ap- 
propriate normalization procedure, the design parame- 
ter set is rewritten as 9 == { G R"" : ||6'||oo < 1 }• 
Suppose also that a gridding approach is adopted. That 
is, for each component 6j, j ~ 1,. . . ,ng of the design 
parameters 6 £ R"", only nc equally spaced values 
are considered. That is, 9j is constrained into the set 



T, = { -1 



2(t-l) 



t = l,...,nc, }. With this 



-3 - \ " ^ ("c,-i) 
gridding procedure, the following finite cardinality set 
e = { [0i,...,e^,f : Oj e T„ J = l,...,ne } is 
obtained. We notice that the cardinality of the set is 
nc = n7=i"'Cj- Another situation in which the finite 
cardinality assumption holds is when a finite number of 
random samples in the space of design parameter are 
drawn according to a given probability, see e.g. [18], [33]. 

The following property states the relation between the 
binomial distribution and the probability of failure under 
this finite cardinality assumption. 

Lemma 3 Suppose that the cardinality of & is no larger 
than nc ■ Then, 

p{N, f], m) < 



nc 



Proof: 



E 

4=0 



N 



v'i^-v) 



N-i 



ncB{N, f], m). 



Denote he < nc the cardinality of 9. Therefore, 9 can 
be rewritten as 9 = {6'(i),6'(2), . . . ,6'("c)}. Then, the 
following inequalities can be easily obtained. 

p{N, rj, m) = PryyNJw G W^ : There exists 9 eO 



than 1 — (5,it suffices to take TV such that nc5(7V,7y, m) < 
i5, where nc is an upper bound on the cardinality of 9. 
As it will be shown next, the required sample complexity 
in this case grows with the logarithm of nc ■ This means 
that we can consider finite families with high cardinal- 
ity and still obtain very reasonable sample complexity 
bounds. 

Theorem 2 Suppose that the cardinality of 9 is no 
larger than nc- Given the nonnegative integer m, 
77 G (0,1) and5£ (0,1), if 



N > inf — I I In — : — h m In a 



a>i rj \a — 1 J \ S 
then p{N, rj, m) < S. Moreover, if 



(9) 



N>iL + \n^ + j2m\n'f] 



then p{N, rj, m) < d. 

Proof: 

From Lemma 3 we have that p{N, rj, m) < S provided 
that B{N, ri,m) < —. The two claims of the property 
now follow directly from Lemma 2 and Corollary 1 re- 
spectively. 



D 



Consider the sampled optimization problem (4) 



N 



min J{9) subject to 2_] ! 



•M) 



eee 



) < m. 



such that E{9, w) < — and E{9) > 77} < 

he 

Y^ Prw" {w G W^ : ^(0(^' , w) < 
i=i 

-andii;(e«)>r;}<nc^ U^(l-7?)^-^< 

1— n \ ^ / 



N 



From the definition oip{N, rj, m) and Theorem 2 we con- 
clude that if one draws N i.i.d. samples {w^^^ , ■ • • , w^^^ } 
from W according to probability Pryv , then with proba- 
bility no smaller than 1 — S, all the feasible solutions to 
problem (4) have a probability of violation no larger than 
r], provided that the cardinality of 9 is upper bounded 
by nc and the sample complexity is given by 



N 



nc 



E . h^i-'?) 



N-i 



N > - (m + \n— + , 2m\n^ 
rj \ V " 



D 



Consider now the optimization problem (4). It follows 
from Lemma 3 that to guarantee that every feasible solu- 
tion 9 G Q satisfies E{9) < rj with probability no smaller 



We remark that taking a equal to the Euler constant, 
the following sample size bound 



N > 



1 



77 \e — 1 



1 "c- , 
in — - — h m 




is immediately obtained from. Theorem. 2. If tti > then 
a suboptimal value for a is given by 



a = 1 



In^ 



In 2^ 



m 



4-3 Optimal robust optimization for design 

In this subsection, we study the so-called scenario ap- 
proach for robust control introduced in [10]. To address 
the semi- infinite optimization problem (2), we solve the 
sampled optimization problem (3). That is, we generate 



N i.i.d. samples {w 



(1) 



,w 



(N) 



} from W according to 



the probability Pryy and then solve the following prob- 
lem: 



min J (9) subject to g{0, w^^^) 



0,^=1, 



,N. (10) 



We consider here the particular case in which J (9) — 
c^6, the constraint g{9, w) = is convex in 6 for all w G 
W and the solution of (10) is unique. These assumptions 
are now stated precisely. 

Assumption 1 [convexity] Let Q C R"" be a convex 
and closed set. We assume that 



j{e) 



:=^ c 9 and g{9, w) 



iff{9,w) <0, 

1 otherwise 



where / : x W — >■ [— oo, oo] is convex in 9 for every 
fixed value ofw G W. 

Assumption 2 [feasibility and uniqueness] For all pos- 
sible multisample extractions {w^^\ ..., w^^'}, the op- 
timization problem (10) is always feasible and attains a 
unique optimal solution. Moreover, its feasibility domain 
has a nonempty interior. 

We now state a result proved in [14], [10] that relates 
the binomial distribution to the probabilistic properties 
of the optimal solution obtained from (10). 

Lemma 4 Let Assumptions 1 and 2 hold. Suppose that 
N , r] G (0, 1) and S G (0, 1) satisfy the inequality 



E 

i=0 



N 



7^'{l-r]y^-' <s. 



(11) 



Then, with probability no smaller than 1 — 6, the optimal 
solution 9n to the optimization problem (10) satisfies the 
inequality E{9n) < rj. 



We remark that the assumptions stated in Lemma 4 can 
be relaxed. Moreover, this result has been extended to 
the possible unfeasible case, see [12], [14] and [15]. 

We now state an explicit sample size bound, which im- 
proves upon previous bounds, to guarantee that the 
probability of violation is smaller than rj with probabil- 
ity at least 1 — S. 

Theorem 3 Let Assumptions 1 and 2 hold. Given rj G 
(0,1) and5€ (0,1), if 



N> inf 



^...^^^-^)(lni + (n.-l)lna) (12) 



iV>-(ln Qj+ne-l + -^2(ne-l)lnij (13) 



then, with probability no smaller than 1 — 5, the optimal 
solution9N to the optimization problem (10) satisfies the 
inequality E{9n) < rj. 

Proof: From Lemma 4 it follows that it suffices to take A^ 
such that B(N, 77, ng — 1) < S. Both inequalities (12) and 
(13) guarantee that B{N,ri,ng — 1) < 6 (see Lemma 2 
and Corollary 1 respectively). This completes the proof. 
D 



We remark that a sample size bound which depends lin- 
early on - is simply obtained taking a equal to the Euler 
constant 



^^K^)(^°^+"^"' 



li ng > 1 then a suboptimal value for a is given by 



a= 1 



5 Sequential algorithms with probabilistic vali- 
dation 

In this section we present a general family of random- 
ized algorithms, which we denote as "Sequential Proba- 
bilistic Validation (SPV) algorithms" . The main feature 
of this class of algorithms is that they are based on a 
probabilistic validation step. This family includes most 
of the sequential randomized algorithms that have been 
discussed in the introduction of this paper. As a matter 
of fact, the non-sequential strategies that can be found 
in the context of Statistical Learning Theory [33] and 
convex scenario [10] can be also provided with an outer 




iterative structure that makes them fit in the proposed 
scheme. We explore this possibihty in Section 8. 

Each iteration of an SPV algorithm includes the com- 
putation of a candidate solution for the problem and a 
validation step. The results provided in this paper are 
basically independent of the particular strategy chosen 
to obtain candidate solutions. Therefore, in the follow- 
ing discussion we restrict ourselves to a generic candi- 
date solution. The accuracy rj e (0,1) and confidence 
S G (0, 1) required for the probabilistic solution play a 
relevant role when determining the sample size of each 
validation step. The main purpose of this part of the pa- 
per is to provide a validation scheme such that it guar- 
antees that for given accuracy rj and confidence 6, all the 
probabilistic solutions obtained running the SPV algo- 
rithm have a probability of violation no larger than rj 
with probability no smaller than 1 — (5. 

We enumerate each iteration of the algorithm by means 
of an integer k. We denote by mk the number of viola- 
tions that are allowed at the validation step of iteration 
k. We assume that mk is given by a function of fc, that 
is, TTife — m{k) where the function to : IN — > IN is given. 
We also denote by Mk the sample size of the validation 
step of iteration k. We assume that Mk is given by a 
function oi k, r] and S. That is, Mk = M{k,ri,S) where 
M:INxRx]R,— ^-W has to be appropriately designed 
in order to guarantee the probabilistic properties of the 
algorithm. In fact, one of the main contributions of [24] 
is to provide this function for the particular case nik = 
for every fc > 1. The functions to(-) and M(-, •, •) are de- 
noted as level function and cardinality function respec- 
tively. 

We now introduce the structure of an SPV algorithm 

(i) Set accuracy ij £ (0, 1) and confidence S E (0, 1) 
equal to the desired levels. Set k equal to 1. 

(ii) Obtain a candidate solution 9k to the robust opti- 
mization problem (1). 

(iii) Set TOfc == TO(fc) and Mk = M{k, t], S). 

(iv) Obtain validation set Vk = {v^^\ ■ ■ ■ , w'^*^)} draw- 
ing Mk i.i.d validation samples from W according 
to probability Prw- 

(v) If ^ g{9k,v^^') < nik, then 9k is a probabilistic 
e=i 
solution, 
(vi) Exit if the exit condition is satisfied, 
(vii) k = k + l. Goto (ii). 

Although the exit condition can be quite general, a rea- 
sonable choice is to exit after a given number of candi- 
date solutions have been classified as probabilistic solu- 
tions or when a given computational time has elapsed 
since the starting of the algorithm. After exiting one 
could choose the probabilistic solution which maximizes 
a given performance index. In the next section we pro- 



pose a strategy to choose the cardinality of the valida- 
tion set at iteration k in such a way that with probabil- 
ity no smaller than 1 — 5 all candidate solutions classi- 
fied as probabilistic solutions by the algorithm meet the 
accuracy 77. 

6 Adjusting the validation sample size 

The cardinality adjusting strategy provided in this sec- 
tion constitutes a generalization of that presented in [24] 
and [17]. To obtain the results of this section we rely on 
some contributions on the sample complexity presented 
in the previous sections. 

We now formally introduce the failure function. 

Definition 3 (failure function) The function fi : 
IN — >■ R is said to be a failure function if it satisfies the 
following conditions: 

(i) n{k) G (0, 1) for every positive integer k. 

00 

(ii) E M(fc) < 1- 
fc=i 

We notice that the function 

^l{k) = 



S,{a)k<' 



where ^(•) is the Riemann zeta function, is a failure func- 

00 
tion for every a > 1. This is due to the fact that E i^ 

i=l 

converges for every scalar a greater than 1 to ^(a). This 
family has been used in the context of validation schemes 
in [13] and in [24] for the particular value a = 2. 

Property 1 Consider an SPV algorithm with given ac- 
curacy parameter rj € (0, 1), confidence 6 G (0, 1), level 
function m(-) and cardinality function Af (■, ■, ■). If there 
exists a failure function /i(-) such that 



m{k) 

E 



M{k,T],S) 



-qHl-'q)^^^-'^'^^-' <5n{k), Vfc>l 



then with probability greater than 1 — 5 all the probabilistic 
solutions obtained running the SPV algorithm have a 
probability of violation no greater than rj. 

The following proof of this property follows the same 
lines as the proof of Theorem 9 in [24] . 

Proof: 

We denote by 5k the probability of classifying at iteration 
k the candidate solution 9k as a probabilistic solution 



under the assumption that the probability of violation 
E{9k) is larger than rj. Then 

^.9(4,w<^')<™fc and £;(^fe) = 77} 









^ / Af (fc,,7,'5) \ ^,^^ _ ^)M(fc,,,.)-^ < ^^(^)^ 



i=0 



Therefore, the probability of misclassification of a can- 
didate solution at iteration k is smaller than Sii{k). We 
conclude that the probability of erroneously classifying 
one or more candidate solutions as probabilistic solu- 
tions is bounded by 

00 00 00 

J2Sk<J2S^,{k)=dJ2Kk)<S. 



k=l fc=l 



fc=l 
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To design a cardinality function M(-,-,-) satisfying the 
conditions of Property 1 we use Corollary 1. 

We now present the main contribution of this part of the 
paper, which is a general expression for the cardinality 
of the validation set at each iteration of the algorithm. 

Theorem 4 Consider an SPV algorithm with given ac- 
curacy rj G (0, 1), confidences G (0, 1) and level function 
m{-). Suppose also that ji{-) is a failure function. Then, 
the cardinality function 

M{k,r],S) = 



M"'^'^^^"^ 



/2TO(fc)ln 



6^{k) 



guarantees that, with probability greater than 1 — S, all 
the probabilistic solutions obtained running the SPV al- 
gorithm have a probability of violation no greater than rj. 

Proof: 

Corollary 1 guarantees that the proposed choice for the 



cardinality function satisfies the inequality 

The result then follows from a direct application of Prop- 
erty 1. D 

We notice that the proposed cardinality function 
M{k,ri,6) in Theorem 4 depends on the previous se- 
lection of the level function m{-) and the failure func- 
tion /i(-). Reasonable choices for these functions are 
m(fc) = la-k\, where a is a non-negative scalar and 
/^(^) ~ ffoAfce, where a is greater than one. We recall 
that this choice guarantees that fJ.{k) is a failure func- 
tion. As shown in the following section, the proposed 
level and failure functions allow us to recover, for the 
particular choice a ~ the validation strategies pro- 
posed in [17] and [24]. In the next corollary, we rewrite 
the generic structure of the SPV algorithm with the level 
function m{k) ~ \ak\ , and state a probabilistic result. 

Corollary 2 Consider the following SPV algorithm 

(i) Set accuracy rj G (0,1), confidence S G (0,1) and 
scalars a > 0, a > 1 equal to the desired levels. Set 
k equal to 1. 

(ii) Obtain a candidate solution 6k to the robust opti- 
mization problem (1). 

(Hi) Set ruk = [afcj and 



Mk 



(iv) Obtain the validation set Vk = {v^^' , . . . ,v^^'''} 
drawing Mk i.i.d. validation samples from W ac- 
cording to the probability Pryy . 

Mfc ^ 

(""J V 12 9((^k,v^^') < rnk, then 6k is a probabilistic 
i=i 
solution, 
(vi) Exit if the exit condition is satisfied, 
(vii) k=k+l. Goto (ii). 

Then, with probability greater than 1—5 all the prob- 
abilistic solutions obtained running the SPV algorithm 
have a probability of violation no greater than rj. 

Proof: The result is obtained directly from Theorem 4 
using as level function m(k) = \ak\ and failure function 

Since the probabilistic properties of the algorithm pre- 
sented in Corollary 2 are independent of the particular 
value of a > 1 , a reasonable choice for a is that minimiz- 
ing the cardinality of the validation samples. In [17] it is 
shown that a = 1.1 is an appropriate choice leading to 
a minimization of the cardinality of the validation sets. 




7 Comparison with other validation schemes 

In this section, we provide comparisons with the vahda- 
tion schemes presented in [24]. This strategy has been 
successfully used in different randomized algorithms 
dealing with uncertain convex problems e.g. in [1], [11], 
[24] . We notice that setting o = and a = 2 in Corollary 
2 we obtain m{k) = for every iteration k and 



M(fc) 






^■^e 



T] V DO 



This is the same cardinality function presented in [24] if 
one takes into account that for small values of 77, — ln(l — 
77) can be approximated by rj. In the same way, a = 
and a = 1.1 lead to the cardinality function presented 
in [17]. 

We noticed that not allowing any failure in each valida- 
tion test makes perfect sense for convex problems if the 
feasibility set 



and s is a strictly positive scalar and t is a non-negative 
integer. 

Proof: 

We notice that a — implies that, at iteration k, the al- 
gorithm classifies a candidate solution 9k as a probabilis- 
tic solution only if it satisfies the constraint g{9k, v^'^') — 
0, /c = 1, . . . , Mk where {w*-^', . . . ,v^^''^} is the randomly 
obtained validating set Vk- Since E{9) > fi for all 6* S O 
and a = 0, the probability of classifying a candidate 
solution as a probabilistic solution at iteration k is no 
greater than 



(1-m) 



< c 



< e 



-^ln(i<^)^ 



Therefore, the probability of providing a probabilistic 
solution at any of the first N iterations of the algorithm 
is smaller than 



e. 



{ 61 e e : g{9, w) = for aU w G W } 



is not empty. Under this assumption, the algorithm takes 
advantage of the validation samples that have not satis- 
fied the specifications to obtain a new candidate solution. 
If Qr is not empty, a common feature of the methods 
which use this strict validation scheme is that a proba- 
bilistic solution (not necessarily belonging to the feasi- 
bility set Qr) is obtained in a finite number of iterations 
of the algorithm, see e.g., [1], [11], [24]. 

A very different situation is encountered when 0^ is 
empty. We now state a property showing that we should 
not use a strict validation scheme (a = 0) to address the 
case of empty robust feasible set because the algorithm 
might fail to obtain a probabilistic solution even if the 
set { 6* e 6 : E{9) < ry } is not empty. 

Property 2 Consider the SPV algorithm presented in 
Corollary 2 with a — Q and a > 1. Suppose that E{9) > 
H > for all 9 G O. Then the SPV algorithm does not 
find any probabilistic solution in the first N iterations of 
the algorithm with probability greater than 



1- 



$(^,[l0g2 7V]), 



where the function $(5, t) is given by 
Hs,t):= 



1 



ifs^l 



t + 1 



othe 



N 

E 

k=l 



^(a)fc° 



S 



e(a) 



^ " / 1 \ ■^ 



k=l 



Taking s — ^^ and using Property 4 in the Appendix 
we have 



N 

E 

fc=i 



1 \ " 



J2-<Hs,\\og,N}). 



k=l 



We conclude that the probability of not finding any prob- 
abilistic solution in the first N iterations of the algorithm 
is smaller than 



1- 



aa) 



$(^,[log2A^]). 



D 



We now present an example demonstrating that a strict 
validation scheme may not be well-suited for a robust 
design problem. 

Example 1 Suppose that 9 = [0,1], W = [-0.08,1], 
77 = 0.1, S ~ lO^'' and that 



g{9,w) = 



if9<w 

1 otherwise. 



Suppose also that Pryy is the uniform distribution. It is 
clear that 9 = minimizes the probability of violation 
and satisfies E{0) = j^ > 0.074. Therefore, we obtain 

E{9) > 0.074 = fx for all 9 e O. 
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Consider now the choice a = 1.1 and a maximum numher 
of iterations N equal to 10® . We conclude from Property 
2 that, regardless of the strategy used to obtain candidate 
solutions, the choice a = and a = 1.1 in Corollary 2 
does not find any probabilistic solution with probability 
greater than 0.98. The choice a ~ 2 leads to a probability 
greater than 0.99. This illustrates that a strict validation 
scheme is not well suited for this robust design problem. 
D 

The next result states that the validation scheme pre- 
sented in this paper achieves, under minor technical as- 
sumptions, a solution with probability one. 

Property 3 Consider an SPV algorithm with given ac- 
curacy parameter rj G (0,1), confidence 6 G (0,1) and 
level function m{-) . Suppose that 

(i) ^{■) is a failure function, 
(ii) The cardinality function M{k, rj, 5) is given by 



M"^^^+^°^ 



/2TO(fc)ln 



Sn{k) 



(Hi) There exist an integer k* , scalars ji £ (0, 1) and 
p G (0, 1) .such that at every iteration k > k* a 
candidate solution Ok satisfying E{9k) < ^ < rj is 
obtained with probability greater than p. 

(iv) lim — jY\ In t-it\ = 0. 

Then the SPV algorithm achieves with probability 1 a 
solution in a finite number of iterations. 

Proof: 

Using the assumption 



fe-s-oo m{k) 5^{k) 



we conclude that 



fc->-oo m{k) 



lim — 

fc— 5-00 rj 



1 



■In 



m{k) Sfi{k) 



■In 



m{k) 6ji(k) 



This implies, along with the assumption fi < rj, that 
there is k such that 



M< 



i(fc) 



M{k)' 



for every k > k. 



We conclude that the SPV algorithm provides candidate 
solutions 9k satisfying 



EiOk) <H< 



m{k) 
M{k) 



(14) 



for every k > max{fc*,A:} with probability no smaller 
than p. The validation test is satisfied if 

M(k) 

Y,9iOkM'^)<m{k), 
1=1 



or equivalently, if 



M(k) 



M{k) 



We notice that 



mll^ikM^')^ 



m{k) 
M{k)' 



M{k) 



M(k) 

^ g{9k,v^^'>) is the empirical 



£=1 

h,v) 



mean associated to g{6k,v). Moreover, recall that the 
probability of obtaining an empirical mean greater than 
the actual probability of violation is smaller than ^ [31]. 
We therefore conclude from (14) that the probability of 
classifying a candidate solution as a probabilistic one is 
no smaller than | for every iteration k > max{fc*,A:}. 
Since | > we conclude that the algorithm obtains a 
probabilistic solution with probability 1. □ 



8 Application to non-sequential randomized al- 
gorithms 

Motivated by the previous results we present an SPV al- 
gorithm which takes advantage of the theoretical results 
obtained in the literature of non-sequential randomized 
algorithms, see e.g., [2]. 

(i) Set accuracy ij G (0,1), confidence 5 G (0,1), and 
scalars a > and a > 1, equal to the desired levels, 
(ii) Set k equal to 1 and Wi equal to the empty set. 
(iii) Set TTifc = [ak \ and 



Mk 



nrik + In 



C(a)fc^ 



2?Tife In 



C(a)fc" 



(iv) Obtain, if possible, a candidate suboptimal feasible 
solution Ok to the optimization problem 



minJ(0) subject to 2_\ 5(^: w) < (card Wfe) 



6»ee 



MiGWfc 



mk 

Mk 



(v) Obtain validation set Vk — {v'^^\ ■ ■ ■ , v^-'^''^} draw- 
ing Mk i.i.d validation samples from W according 
to probability Pr-w. 
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(vi) If a feasible solution 9k was found at step (iv) then 
classify it as a probabilistic solution if 



E 

i=i 



giOk^v^'^'^) <mk. 



(vii) Exit if the exit condition is satisfied, 
(viii) Wfc+i = Wfc U Vfc, fc = fc + 1. Goto (iii). 

Under rather general assumptions like finite VC- 
dimension of the binary function g{-^ •), see [2] for further 
details, it follows that the feasible solutions obtained at 
step (iii) of the algorithm have a probability of violation 
smaller than ^|^ < 77, with a probability that tends to 
1 with the cardinality of Wk ■ From this observation we 
conclude that if the optimization problem at step (iii) is 
feasible with probability greater than zero, the proposed 
algorithm satisfies the assumptions of Property 3 and 
therefore provides a probabilistic solution that meets 
accuracy 77 and confidence 6. The main advantage of the 
proposed algorithm with respect to the non-sequential 
algorithms available in the literature is that no explicit 
bound on the number of samples is required. This might 
lead to a substantial reduction of the number of re- 
quired samples in particular in the case of non-convex 
uncertain problems. 



where w gW. 

The uncertainty set W is 

yv={w^[t A B]^,ie [0,1], Ae [1,3], Be [1,3]} 

For a given order d, we define the regressor (p^ : W — i- 
R'^+i as 



Mw)^M[t A Bf)^ 



Itt' ■■■t'' 



The objective of this example is to find a parameter 
vector e = Yld: XdV^ Id e E.'*+^ and Arf G W^+^ such 
that, with probability no smaller than 1 — 5, 

Prw{ w e W : \v{w) - l'^Vd{w)\ > X^\ipd{w)\ } < -q. 

The vector |(^d('w)| is obtained from the absolute values 
of ipd{w). The binary function g : x W — J- {0, 1}, is 
defined as 



g{e,w):-- 



if meets design specifications for w 

1 otherwise. 



9 Numerical example 

The objective of this numerical example is to obtain 
probabilistic upper and lower bounds of a given time 
function y : W — s> R of the form 



1 3 

j/(w) = [A{1 + -t^) sin(7i + 0.5) + B]e-5*, 



4[-..,::; ._ .^. . 

2 - 



Finite famiiies soiution 



'•'-•■::::i::r^-5;s;:::r... 



0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 
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Aigorilhm soiution 
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Fig. 1. Initial data set and envelope of the set of solutions. 



where "design specifications" means satisfying the fol- 
lowing constraint: 

\y(w) - 7j(/3d(w)| < \l\ipd{w)\ 

for randomly generated samples w e W. 

A similar problem is addressed in [14] using the sce- 
nario approach. For the numerical computations, we take 
5 — 10^^ and 77 = 0.05. We address the problem from 
the finite families, scenario and SPV approach, and use 
the explicit sample complexity derived in the previous 
sections. 

9. 1 Finite families approach 

We apply the results of Section 4.2 to determine both 
the degree d and the parameter vectors (7d,Ad) that 
meet the design specification and optimize a given per- 
formance index. 

In this example, as it will be seen later, a finite family of 
cardinality nc = 150 is considered. In order to compare 
the finite family approach with the scenario one, we con- 
sider no allowed failures (i.e m — 0). For this choice of 
parameters {m = 0, nc = 150, S — 10^^ and 77 = 0.05), 
the number of samples N required to obtain a solution 
with the specified probabilistic probabilities is 377 (see 
Property 2). A set D oi M — N samples is drawn (i.i.d.) 
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from W. We use these samples to select the optimal pa- 
rameters {"fd, ^d) corresponding to each of the different 
regressors fdi')- Each pair (7^, A^) is obtained minimiz- 
ing the empirical mean of the absolute value of the ap- 
proximation error. That is, each pair (7^, A^) is the so- 
lution to the optimization problem 



.^i? i E ^Jl^'^HI 



weD 



0.8121. Figure 1 shows the approximation for the set V 
and the obtained probabilistic upper and lower bounds 
for the random function. 

Finally, for illustrative purposes, we check with a vali- 
dation set of sample size Ny — ION, obtaining a num- 
ber of 60 failures. The experimental violation probability 
turned out to be ?7oxp = 0.0146, while the specification 
was T] = 0.05. 



s.t. \y{w) - '-ijipd{w)\ < X^\ipd{w)\, Vw e D. 9,2 Convex scenario approach 



We notice that the obtained parameters do not neces- 
sarily satisfy the probabilistic design specifications. In 
order to resolve this problem, we consider a new set of 
candidate solutions of the form 



e = {0d,, = (7d,e(^-i)Ad) 



^ ^T • • • ^ ^inax ) .7 ^ 7 • • • 7 Jinax / • 



In this case we take advantage of the result of Subsection 
9.1 and take d = 5 as the order of the approximation 
polynomial. Following the scenario approach we draw a 
set Wfc of N samples (i.i.d) from W and solve the convex 
optimization problem 

min \lE{\^d{t)\} 
s.t. \y{w) - jj(pd{w)\ < Xj\ipd{w)\, Vw e Wk. 



This family has cardinality Uc = rfmaxjmax- We take a 
large factor e^T""^-', to increase the probability of meet- 
ing the design specifications. Therefore, choosing a large 
enough value for jmax leads to a non-empty intersection 
of 8 with the set of parameters that meet the design 
specifications. In this example, we take jmax = 15 and 
rfmax = 10. This yields to ric = 150. 

Using the finite family approach, we choose from O the 
design parameter that optimizes a given performance 
index. We draw from W a set V oi N (i.i.d.) samples 
and select the pair that minimizes the empirical mean 
of the absolute value of the approximation error in the 
validation set V. That is, we consider the performance 
index 

-E 

toGV 

subject to the constraints 



e(^-i)Aj|^,H| 



\v{w) -7rf^d(w^)| 



<e(^-i)Aj|^<i(w)|, VweK 



We remark that the feasibility of this problem can be 
guaranteed in two ways. The first one is to choose jmax 
large enough. The second one is to allow m failures. As 
previously discussed, in this example we take jmax = 15 
and TO = 0. 

As the cardinality N oiV has been chosen using Prop- 
erty 9, the probability of violation and the probability 
of failure of the best solution from are bounded by r\ 
and 8 respectively. 

The obtained solution corresponds to d = 5 and j = 13. 
The corresponding value for the performance index is 



In order to guarantee the design specifications we use 
Property 10 to determine the value of iV. Since the 
number of decision variables is 1(d -I- 1), 77 = 0.05 and 
(5 = 10^®, the resulting value for N is 845. We notice 
that the convex scenario approach does not apply di- 
rectly to the minimization of the empirical mean. This 
is why one has to resort to the exact computation of the 
mean of the approximation error Aji^^d^s^l^)!}, see [14]. 

Figure (1) shows the initial data set generated using 
the procedure described above, plus the envelope that 
contains all the solution polynomials. 

Again, for illustrative purposes, we check with a valida- 
tion set of size iV„ = lOA^, obtaining zero failures. The 
experimental value ?7oxp = is obtained, while the spec- 
ification was r\ — 0.05. 

Using this strategy, 845 design data are required, bigger 
than the number of required samples to use the finite 
families approach. We obtained a performance index of 
0.8748, slightly larger than that obtained by the finite 
families strategy. The advantage of the finite families 
approach is that, using a smaller number of samples, a 
similar performance is obtained. This allows us to deter- 
mine the best order of the polynomial with the further 
advantage that the exact computation of the mean of 
the error is not required. 

9.3 SPV algorithm 

We again take advantage of the result of Subsection 9.1 
and take d = 5 as the order of the approximation polyno- 
mial. Following the SPV algorithm approach, we begin 
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setting T] = 0.05, confidence S = 10^^, scalars a — 0.75, 
a = 2 and iteration index k ~ 1. The initial Wfc is a set 
of 400 samples drawn from W according to probability 
Prw 

(i) A candidate solution Ok to the problem 

™in AjV|(/?rf(i)| 
s.t. \y{w) - 7j(^d(w;)| < j-2Aj|(/?d(w)|, Vw e Wfc 



f? 


Nfinite 


-^^ convex 


Nspv 


0.2 


190 


212 


582 


0.1 


378 


423 


543 


0.05 


754 


845 


686 


0.02 


1884 


2113 


2230 
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is obtained, 
(ii) Set TTife = [a(fc)J and 



(iii) 
(iv) 



(v) 
(vi) 



Mk 



- mfc + In ; 
V \ 



2mi. In 



C(Q:)fc" 



Obtain validation set Vk = {v^^\ ■ ■ ■ , v^-'^''^ draw- 
ing Mk i.i.d. validation samples from W according 
to the probability Prw- 

Mfc 

If Y^ g{dk,v^^^) < rrik, then 6k is a probabilistic 

£=1 

solution. The failure function is g{6k,v^^^) 

roif|yH-7j^,H|<Aj|^dHI 

[ 1 otherwise. 

Exit if the exit condition is satisfied. 
k = k + l.Wk = Wk{jVk. Goto {i). 



Figure (1) shows the initial data set generated using the 
procedure described above, and the envelope that con- 
tains all the solution polynomials. Using this strategy, 
686 data are required. We obtained a performance index 
of 0.8877, slightly larger than that obtained by the finite 
families strategy. 

The number of failures obtained in the last step of the 
algorithm is m = 5, being the empirical probability of 
failure r; = ^ = < 0.05. 

We remark that if we set a = in the algorithm, there are 
no allowed failures and this coincides with the approach 
studied in [24]. In this case, the algorithm did not find a 
solution for rj = 0.05 and Mj, < 30000. This is consistent 
with the results on Section 7. 

In Table 1 the results of the three approaches are com- 
pared for different values of rj. 



In this paper, we have derived sample complexity results 
for various analysis and design problems related to un- 
certain systems. In particular, we provided new results 
which guarantee that a binomial distribution expression 
is smaller than a pre-specified value. These results are 
subsequently exploited for the analysis of worst-case per- 
formance and constraint violation. With regard to de- 
sign problems we considered the case of finite families 
and the special case when the design problem can be re- 
cast as a robust convex optimization problem. 

We also presented a general class of randomized algo- 
rithms based on probabilistic validation. We provided a 
strategy to adjust the cardinality of the validation sets 
to guarantee that the obtained solutions meet the prob- 
abilistic specifications. The proposed strategy is com- 
pared with other schemes from the literature and it has 
been shown that a strict validation strategy in which the 
design parameter has to satisfy the constraints for all 
the elements of the validation set might not be appro- 
priate in some situations. We proved that the proposed 
approach does not suffer from this limitation because 
it allows the use of non strict validation test. As it has 
been shown in this paper, this relaxed scheme allows us 
to reduce, in some cases dramatically, the number of it- 
erations required by the sequential algorithm. Another 
advantage of the proposed approach is that it does not 
rely on the existence of a robust deterministic solution. 
Furthermore, the presented strategy is quite general and 
is not based on a convexity assumption. 
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11 Appendix 
Proof of Corollary 1 

We first consider ttie case m = 0. Then, we obtain 
B{N,r],0) = (1 - 7])^ = e^i"(i^'') < e"''^. Tiierefore, 
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it follows from rjN > In |- that c '/^ < e'" "^ = 8. This where, given s > and the integer t > 0, 

proves the result for m = 0. 

■ — -. — 7;t^ */s7^l 

Consider now the case m > 0. We first prove that , , ' 

$(s,i):= 



1-21 



h{r) := v/2(r-l) - In fr + ^2(^-1)) > 0, Vr > 1.(15) 



t+1 



otherwise. 



Since h{l) = 0, the inequality h(r) > holds if the 
derivative of h(r) is strictly positive for every r > 1. 



dr 



h{r) 



1 + 



^2{r - 1) r+v/2(r-l) \ ^2(^-1) 



1 \ /^ 1 + vX^^ 

V2F^TyM r + v/2(f^Ty^ 



Proof: Given TV > and s > 0, define i := [log2(A^)l 

2* 

and 5(t) := X) aF- With these definition we have 



k=l 



N 2' 



fc=l fc=l 



In what follows we show that S{t) < 1 + i^-^Sit ~ 1) 
for every integer t greater than 0. Since 5(0) — 1 and 
S{1) = l + 2~*, the inequality is clearly satisfied for 
t = 1. We now prove the inequality for t greater than 1 



r- 1 



> 0, Vr > 1. 



\^2{r^l)J \r+^2{r-l)^ 

This proves the inequality h(r) > 0, for all r > 1. Denote 
now a = r + \j2ir — 1), with r ~ 1 + ^ In |. Clearly 
a > 1. Therefore, from a direct application of Lemma 2, 
we conclude that it suffices to choose N such that 



Nr\ > In — + ?7i In a ) = 

a — 1 V 



- 1 + ^/2(r - 1) V V V V J J 



r- t + V2(^ 
Since /i(r) > we conclude that 



r-l + ln(r+ v/2(r-l)) _^ 
r-l + ^2(r-l) 

From this inequality, we finally conclude that inequality 
B(iV,r/,m) <(5holdsif 

Nri > {r + v 2(r — l))m = m + In — + W 2mln — . 



D 



Property 4 Suppose that N is a positive integer and 
that s is a strictly positive scalar. Then, 



^ 1 



2* -. 2*"^ 






{2ky {2k- ly 



2-« Vl + V — 

^^ k^ ^^ (2k-l 



fe=i fe=i 



qT — ± 

:2-5(.-i)+i+5:^ 



r,t-l 



<2-^5(.-l) + l + E^ 



fe=2 



fe=2 ^ 



:2-5(t-l) + l + 2- 5^" 1 



fe=l 



<2-5(t-l) + l + 2-^l 



fe=i 
= 2-"5(t-l) + l + 2-'^5'(i-l) 
= l + 2i-''5'(f-l). 

We have therefore proved the inequality S{t) < 1 + 
2^^''S{t—l) for every integer t greater than 0. Using this 
inequality in a recursive way with S'(O) = 1 we obtain 



S{t) <^2(i-'')'= = $(s,t). 



fe=i 



k=0 

This proves the result. □ 
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